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ABSTRACT 

£tate vocational rehabilitation agencies nead a 
system for effectively and efficiently collecting, processing, and 
reporting program information. Design of such a system begins with a 
thorough understanding of the nature of the information needs and the 
particular applicat ions for which the inf ormat ion will be used. 
Information needs may be divided into two categories: those arising 
from the requirements of external audiences that influence the 
program and those emanating from the internal demands of planning, 
management, control, and operation. When designing a state vocational 
rehabilitation agency management information system, each of five 
system operations should be addressed: data collect ion and 
preparation, data entry, data storage and retrieval, central 
processing, and report preparation. Some common issues and concerns 
for state agency inf ormat ion systems include the use of only a 
portion of the data rout inely collected for agency planning, 
management, or operations; the maintenance by field staff of data 
routinely recorded in a centralized data base; the creation of 
reports that are not designed by identifying the part icular need they 
are to fulfill and contain unnecessary inf ormat ion; and the marginal 
attempts to highlight and interpret significant information in a 
report . (Attachments describe efforts in Virginia and Illinois to use 
existing program data to analyze agency performance through a case 
load analysis system.) (YLB) 
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ABSTRACT 



A CONCEPTUAL APPROACH TO THE DESIGN OF MANAGEMENT INFORMATION SYSTEMS FOR 
STATE VOCATIONAL REHABILITATION AGENCIES. 

Report by Stanley E. Portny, Stanley E. Portny and Associates, Inc., 
June 1982, 24 pp. 

This paper establishes a conceptual framework for assisting state 
vocational rehabilitation agencies to effectively collect, process, and 
report program information, and to utilize this information to carry out 
their diverse roles and responsibilities. The report discusses the need 
for information to meet the needs of the external audiences which influence 
the program, as well as the need for information to meet internal planning, 
management control, and operation requirements. After describing the over- 
all structure of a state VR agency management information system, the report 
^describes an approach to the design of an MIS and describes common issues 
and concerns for most state agency information systems. Attachments des- 
cribe efforts in several states to use existing program data to analyze 
agency performance through a "Caseload Analysis System. 11 
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I. INTRODUCTION 

The Vocational Rehabilitation (VR) program is a state-federal partner- 
ship established to assist disabled individuals to prepare for and engage 
in gainful employment. The Rehabilitation Services Administration, the 
federal program agency, coordinates the disbursement of federal VR funds 
to state agencies; develops general program policies, regulations and 
guidelines; and provides evaluation, technical assistance, and training 
support. State VR agencies are responsible for refining and interpreting 
program policies, developing operating procedures, allocating and managing 
the disbursement of federal and state program funds, providing or coordinating 
the provision of services to clients and monitoring a variety of special grant 
and project activities designed to support the program. 

To accomplish its ultimate goal of effectively providing rehabilitation 
services to eligible disabled people, the state VR agency must successfully 
perform a diversity of roles in a complex program environment. On the one 
hand, it must negotiate and collaborate with RSA, the state legislature, 
the governor's office, and other federal and state programs which provide 
similar services to the VR target population. Additionally, it must at all 
times work closely with disabled people themselves to ensure that the pro- 
gram is being responsive to the special needs of the target population. On 
the other hand, it must develop and operate an effective organizational 
structure for planning and coordinating the actual delivery of services to 
its clients. 

Successful performance of these numerous activities requires that sub- 
stantial amounts of program information be provided to selected audiences. 
Design of a system for effectively and efficiently collecting, processing, 
and reporting this information begins with a thorough understanding of the 
nature of these needs and the particular applications for which the infor- 
mation will be used. 
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II. STATE VR AGENCY INFORMATION NEEDS 

State VR agency information needs may be divided into two categories: 
1) those arising from the requirements of external audiences that in some 
way exert power over or influence the program; and 2) those emanating from 
the internal demands of planning, management control, and operation. 

A. EXTERNAL REQUIREMENTS 

Information for external audiences is most frequently required to con- 
firm program compliance with established policies and guidelines, to justify 
budget requests and legislative initiatives and to demonstrate program 
accountability. Additionally, the agency may have to respond to requests 
for information from other programs and the general population which des- 
cribes program purpose, the target population, services provided, and pro- 
gram performance. The task of the agency is to respond to these information 
needs by making optimal use of data already available and, thereby* to 
minimize the need for special data collection and/or analysis activities. 

B. INTERNAL REQUIREMENTS 

1. Planning, Management Control, and Transactions in the State VR Agency 

Internally, program information is required to facilitate the following 
organizational activities: 

• planning the establishment of program policy (i.e., mission, 
goals, objectives, and operating procedures) and the develop- 
ment of performance targets; 

• management control — the direction and coordination of 
activities and resources to achieve specific results; and 

• events or transactions — the actual provision of services, 
transfer of resources, and other related activities.- 

These activities are regularly performed on an ongoing basis; the cyclical 
nature of the interrelationships among them is illustrated in Figure 1. 
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Figure 1 

Activities Conducted in the Operation 
of a State VR Agency 
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Strategic planning entails the formulation of program policy, in response 
to the need for program services , program capacities, and external resource 
and environmental constraints; operational planning consists of translating 
the program goals and objectives into measurable performance targets. 
Management control is accomplished through the comparative assessment of 
program performance with the performance targets. Program events and trans- 
actions are conducted in accordance with established guidelines, operating 
procedures, and accepted principles of good practice. 

A substantial amount of information on program operations, as well as 
on the external environment, is required to support these organizational 
activities. Information used to facilitate program events and transactions, 
which, in general, entail decisions of a highly structured nature that are 
made in accordance with specified sets of rules, must be accurate, well- 
defined, and in prespecified formats. On the other hand, the information 
requirements for planning, which consists of decisions that depend more upon 
judgment and analysis of particular situations, are for flexible access to 
a data file and the performance of analyses which cannot be totally speci- 
fied up front. Information requirements for management control tend to lie 
sonewhere between these two extremes. A table which depicts salient charac- 
teristics of the information required for planning, management control, and 
operations is presented in Figure 2. 

Four types of data are collected and used in the state VR agency: 

• transaction data, 

e system descriptors, 

• performance targets, and 

• environmental characteristics. 

Transaction data either serve as a basis upon which operational decisions 
are made and resulting actions taken, or describe program actions which 
will be or have been taken. Examples include the type and cost of services 
authorized for ? client and the data on which a client begins or ends a 
particular service. System descriptors include characteristics of the ser- 
vice delivery programs, resources used, or clients served. 

Performance targets are measurable objectives which are to be achieved 
in a specified time period. Targets are most frequently established for 
system descriptors, such as the number and characteristics of clients to be 
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Figure 2 H 
Characteristics of Information Requirements 
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served, and for aggregated transaction data, such as the amount of program 
funds to be expended or the average amount of time to be spent in the pro- 
gram. Environmental characteristics include descriptions of entities and 
situations external to the state VR agency which impact upon or are of 
importance to program operations. Examples include descriptions of other 
programs which offer services similar to those provided by the agency, un- 
employment and other job market statistics, and the projected federal VR 
budget for the next fiscal year. 

typically, transaction data, selected system descriptors, and selected 
environmental characteristics are required to support program events and 
transactions. Management control entails the comparison of program perform- 
ance, as represented by aggregated transaction data and system descriptors, 
with established performance targets. Planning involves the use of all four 
types of data. 

2. State VR Agency Operational Subsystems 

Typically, the state VR agency is comprised of the following operational 
subsystems: 

• client services, 

• indirect services and support, 

• funds , 

• vendors and faciltities, and 

• personnel . 

Client services include the assessment of program eligibility, the develop- 
ment and administration of service delivery programs for each client and 
the helping of clients to obtain and maintain gainful employment. Indirect 
services include projects to improve service delivery capability, as well 
as to reduce physical and attitudinal barriers to disabled people in the 
community. 

Management of funds entails arranging for or actually performing the 
disbursement of program dollars for goods and services obtained. Manage- 
ment of vendors and facilities includes arranging for and procuring the 
goods and services prescribed for clients in their individualized rehabili- 
tation program. Administration of state agency personnel encompasses their 
recruitment, assignment, and support. 
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Individual administrative structures and/or organizational units are 
established to conduct these activities and to manage these program resources. 
However, successful operation of. one subsystem depends upon close coordination 
with one or more of the others. 

Key interrelationships among these five operational subsystems are 
depicted in Figure 3, As illustrated, agency personnel receive salaries 
and iv e program funds to purchase goods and services from vendors and facili- 
ties, in order to assist clients to obtain and maintain gainful employment 
and to reduce their dependency upon society. In addition, program personnel 
work to develop new service delivery capacity, as well as to reduce environ- 
mental barriers to disabled people in the community. 

Program funds actually flow to personnel (to pay their salaries) . and 
to vendors and facilities (to pay for services and equipment obtained). The 
numbers and types of agency staff maintained depend upon the amount of 
program funds available for services, as well as the number of clients to 
be served and the types and extent of services to be provided. On the other 
hand, the number of clients that can be served and the types of services 
which may be authorized and provided depend upon the amount of funds avail- 
able, the number and types of vendors and facilities available, and the num- 
bers and capabilities of available staff. 

3. Information Requirements for Planning, Management Control, and Operation 
of Individual Agency Subsystems 

In the normal operation of the state VR agency, planning, management 
control, and the conduct of the daily transactions occur in each of the 
five subsystems noted. The particular objectives of these activities are 
presented in Figure 4. 

Information required to support the conduct of transactions* in each of 
the subsystems may be discerned directly from the description of the activity 
being performed. Client services transactions typically require descriptive 
data and professional assessments on the client, as well as a working know- 
ledge of rehabilitation techniques and available resources. Coordination 
and utilization of funds, vendors and facilities, and personnel requires 
descriptive data on the planned action, procedural guidelines describing the 
constraints applying to such a transaction, and possibly approvals allowing 
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Figure 3 

Interrelationships Among Major Subsystems 
in State VR Agency Operations 
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Figure 4 

Objectives of Planning, Management Control , and Transactions 
in the Individual State VR Agency Subsystems 



SUBSYSTEM 


PLANNING 


MANAGEMENT CONTROL 


TRANSACTIONS 


CLIENT 
SERVICES 


0 To assess the size and 
service needs of the 
target population 

• To develop improved 
methods for serving the 
target population within 
proj ected resource 
constraints 

• To develop a service 
delivery plan. 


• To insure prescribed ser- 
vices are provided effec- 
tively, efficiently and 
in a timely manner in com- 
pliance with Federal and 
State requirements 
(Caseload Management) 

• To monitor client 
progress in response to 
the services provided 
(Caseload Management) 

• To insure agencywide ser- 
vice delivery goals and 
objectives are met 
(Program Management) 


• To determine client 
eligibility 

• To develop a service 
plan (IWRP) 

• To provide or arrange 
for the provision of 
prescribed services 

• To help the client to 
obtain and maintain 
gainful employment 


INDIRECT 
SERVICES 


e To determine components 
of the service delivery 
system where increased 
capability is required 

• To identify attitudinal 
and physical barriers 
to the disabled popula- 
tion in the community 

• To select projects to 
be undertaken by the 
agency. 


• To insure that projects 
are performed within 
established time sche- 
dules and achieve the 
results promised. 


• To perform the tasks 
and activities detailed 
in all projects which 
fall under this cate- 
gory. 
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SUBSYSTEM 


' , PLANNING 


MANAGEMENT CONTROL 


; TRANSACTIONS 


FUNDS 


i To project the expected 
sizes of future Federal 
and State VR program 
budgets 

• To estimate the cost of 
providing particular 
services to specific 
types of clients 

• To identify alternative 
sources of funds or 

. similar benefits 

• To develop a program 
budget. 


• To insure that annual ex- 
enditures closely approx- 
imate but do not exceed 
available Federal and 
State program funds 

i To insure that al2 re- 
sources in addition to 
VR program dollars are 
used to the maximum 
extent possible. 


• To authorize the obli- 
gation of funds for a 
particular good or 
service 

9 To authorize payment 
of a voucher or invoice 

• To prepare a check for 
payment (sometimes). 


VENDORS AND 
FACILITIES 


• To estimate the projected 
need for vendors and 
facilities of different ^ 
goods and services 

• To develop a plan for 
insuring the availability 
of appropriately quali- 
fied vendors and 
facilities, 


• To insure that goods and 
services are provided by 
vendors and facilities 
in a cost effective and 
timely manner. 


• To select a vendor or 
facility to provide a 
particular good or 
service 

• To arrange for the good 
or service to be 
provided 

• To certify that the . 
good or service has 
been provided. 


PERSONNEL 


• To estimate the projected 
need for State agency 
staff in the future 

• To identify sources from 
which staff will be re- 
cruited, when necessary 

• To develop a plan for 
recruiting required 
staff. 


• To insure that author- 
ized positions are filled 
with appropriately quali- 
fied staff 

• To insure that staff are 
adequately performing the 
jobs for which they are 

• responsible. 


• To hire State agency staff 
o To assign staff to appro- 
priate organizational units 
» To provide required 
training to staff. 



the transaction to be made. The information to facilitate indirect 
service activities depends entirely on the paiticular project or initia- 
tive being conducted. 

The objectives of state VR agency planning and management control are 
more generalized, however, and the information required to support them is 
not derived as easily. In Figure 5, a table is presented which indicates 
information which would help to address the particular objectives for each 
agency subsystem as noted in Figure 4. 
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Figure 5 

Selected Information Requirements for Planning and Management Control 
of State VR Agency Organizational Subsystems 



PLANNING 



• Definition of the target population to be 
served 

• -Size and characteristics of the target 

population potentially available for 
services 

o Service needs of the target population 
which the State VR agency will attempt 
to address 

• Estimates of the number and characteristics 
of people who will apply for services 

t Estimates of the percentage of applicants 
that will be eligible to participate in the 
program 

• Effectiveness/accuracy of the eligibility 
determination process 

• Effectiveness of different services 

o Capacity of the program to accept and 
effectively serve new clients 

• Current and projected availability of 
required goods and services 

o Availability of other programs potentially 
able to provide same or similar services 
to VR target population 

• Proportion of clients successfully comple- 
ting the program 

-'Proportion-of-clients-not-successfuHy^ — - 
completing the program and the reasons 
therefore 

• Proportion of clients successfully closed 
that are placed in specified employment 
situations 



MANAGEMENT CONTROL 



Caseload Management 

ii Amount of time client has been 
in the program 

• Amount of time client has been 
in current status 

• Total improvement client has 
shown to date since entering 
the program 

• Extent to which service cur- 
rently being provided to client 
is achieving/has achieved its 
intended purpose 

• Characteristics of employment 
situation at closure ' 

i For clients placed in gainful 
employment, the success which 
they are having in their posi- 
tion 

Program Management 

e Number of client applications 
received 

• Number of applicants referred 
by different sources 

• Number and characteristics of 
clients- determined eligible 

• Number and characteristics of 
clients determined ineligible 

• Relative frequencies of dif- 
ferent reasons for ineligibility 
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Figure 5 (continued) 



SUBSYSTEM 


PLANNING 


MANAGEMENT CONTROL : 




• Nature and magnitude of benefits realized 
by clients as a result of program parti- 
cipation 

• Political support or opposition to VR 
program in the State and nationally 

- • Job market situation in the State 


i Number and characteristics of 
clients served 

• Number and characteristics of 
clients successfully rehabili- 
tated • 

• Number and characteristics of 
clients not successfully rehab- 
ilitated 

• Relative frequencies of differ- 
ent reasons for unsuccessful 
rehabilitations 

i Number of clients in the vari- 
ous stages of the VR program 

o Average time spent by clients 
in the various stages of the 
VR program 

• Average gain or improvement in 
areas of critical life function- 
ing realized by clients as a 
result of program participation 

• Extent to which clients retain 
over time the benefits realized 
through program participation 

• Types and number of units of 
services provided 
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SUBSYSTEM 


PLANNING 


MANAGEMENT CONTROL 


INDIRECT 
SERVICES 


• Projected status of legislative provisions 
and funding for special project activities 

i Prioritized list of service delivery systeir 
needs and potential projects with the 
community 


• Time schedule and milestone 
chart for each project or 
activity 

* 

% Periodic assessment of project 
progress 


FUNDS 


• Projected Federal and State program budgets 
for the next one to five years 

• Projected increases/decreases in the costs 
of services 

• Projected sources of similar benefits and 
the amount of funds potentially available 

• Projected changes in the average cost of 
serving a client, successfully rehabilita- 
ting a client 

i Existing future obligations for services 

• Cost-effectiveness of different service 
delivery alternatives 


Caseload Management 

• Expenditures to date for 
each client 

, • Percentage of total expendi- 
tures planned for each client 
which have been expended to 
date 

• Outstanding authorizations for 
each client 

• Amount of similar benefits 
planned and utilized for each 
client 

Program Management 

• Total amount of Federal, State 
and other funds available for 
the year 

• Expenditures to date in each of 
the identified budget categories 

• Total outstanding authorizations 
for goods and services 

• Total amount of similar benefits 
utilized 

a case that is paid for with 
similar benefits 
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SUBSYSTEM 



VENDORS AND 
FACILITIES 



PERSONNEL 



PLANNING 



Projected demand for individual goods and 
services 

Current availability of needed goods and 
services 

Projected increases /decreases in the 
number and/or capacity of vendors and 
facilities 

Cost-effectiveness of different vendors 
and facilities 
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Projected staff attrition rates 
Potential sources of trained staff 
Projections of qualifications which 
staff needed will have to possess 
Current state personnel recruitment 
policy and procedures and expected 
changes 

Estimates of the number of staff required 
to provide different levels of specified 
services 

Amount of funds available for staff in- 
service training 



Total number of clients currently 
being served by each facility 
available to serve clients 
Total number of clients served year- 
to-date by each facility 
Percent of all clients receiving a 
particular good 'or service that went 
to each of the vendors or facilities 
offering those goods or services 
Number and percentage of clients 
served by each facility who successfully 
completed the program : 
Reasons for client dropout short of 
completing the program at each facility 
For each facility, the average gain 
realized by clients as a result of 
services received 

Average length of time required to suc- 
cessfully complete the program at a 
facility 

Total amount of program funds expended 

for the year to date at each individual 

vendor and facility 

Average cost of particular goods or 

services at each available vendor and 

facility 



e Number of existing vacancies in the 
agency 

• Average length of time to fill a vacancy 

• Average and distribution of performance 
ratings for agency staff 

• Staff turnover and attrition 
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III. STRUCTURE AND DESIGN OF A STATE VR AGENCY 
MANAGEMENT INFORMATION SYSTEM 

A. ELEMENTS OF AN MIS 

A Management Information System is an integrated, man/machine system 
for providing information to support planning, management, and operating 
functions in an organization. The information activities may be large- 
scale and complex or they may be few and simple; the system may be exten- 
sively documented or completely informal; and the system may utilize the 
most technologically advanced computer support or be completely manual. 
In any case, the MIS will include the following five operations: 

• data collection and preparation, 

• data entry, 

• data storage and retrieval , 

• central processing, and 

• report preparation. 

1 . Data Collection and Preparation 

Information is collected in the state VR agency by the following 
methods: 

• regularly scheduled data submissions, 

• ad hoc data submissions, 

• special studies and investigations, 

• developmental planning activities, and 

• miscellaneous information-gathering. 

All transaction data and most system descriptors are recorded on stan 
dardized input forms at specified times in the rehabilitation process. Ad 
hoc data submissions and special studies may be designed to elicit more 
information on particular system descriptors and aspects of program perfor 
mance, while performance targets are developed through specialized plannin 
analyses and activities. Information about the program environment and 
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other isolated topics of interest is obtained through specially designed 
data collection activities * 

2. Data Entry 

In noncomputerized information systems, entry of data may simply entail 
placing data collection forms or tally sheets which summarize the raw data 
in a notebook or a file drawer or sending them to a staff person responsible 
for their maintenance. In computerized systems, the data must be trans- 
ferred from the raw data collection forms or tally sheets to a computerized 
data file. This process may be accomplished either by having the raw data 
forms and/or tally sheets routinely submitted to a central location (the 
same as for noncomputerized systems) and then entered into the computerized 
data file by one or more specially designated agency staff or by having data 
entered directly into the computer on remote terminals by staff responsible 
for its collection and /or preparation. 

Most often, data entry will entail some combination of these various 
approaches. 

3. Data Storage and Retrieval 

Information collected in the state VR agency is stored in one or more 
of the following locations: 

• central computerized data file(s), 

• centralized manual files in the state office, 

• centralized manual files in the field offices, and/or 
o manual files belonging to agency staff. 

If the state has computerized some or all of its information system, . 
most frequently the data base established has two principal components: 
1) a statistical file, and 2) a fiscal file. 

4. Central Processing 

Processing of raw data collected and stored in the information, ;-/*^om 
entails the tallying, combining, and analyzing of the numbers in order 
describe performance clearly and concisely, to determine trends in program 
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performance and, where possible, to suggest reasons for observed behavior. 
In noncomputerized systems, processing is performed by hand, while processing 
in computerized systems is accomplished with specially developed programs 
and a wide array of existing software packages. 

5. Report Preparation 

Information can be prepared and presented in the following forms: 

• transaction documents, 

• preplanned reports, 

• preplanned inquiry responses, 

• ad hoc reports and inquiry responses, and 

• man/machine dialogue. 

Transaction documents include such items as service authorizations, 
vendor invoices, IWRPs, client case files, and so forth. The transaction 
document serves one of two purposes. Either it describes or confirms an 
action that has been or will be taken, or it can serve as a request for 
action or provide instructions for taking the action. 

Preplanned reports are produced in accordance with an established 
format and are generated at regularly scheduled intervals. They either 
describe certain program conditions at the end of the time period in question 
or summarize activities and results which have occurred during that time 
period. 

Preplanned inquiries are questions about performance which may be 
asked for which the computer software required to generate the answers has 
already been developed, but about which regularly scheduled reports are not 
routinely produced. Preplanned inquiries are addressed as they occur, in 
accordance with established reporting formats. 

Ad hoc reports and inquiry responses are those information outputs 
which are produced in response to a particular request, in formats speci- 
fically designed for the situation at hand. Information to respond to such 
requests may be obtained from the data base already in existence, or new 
data may have to be especially collected. 

Man/machine dialogue entails the direct use of a computerized model 
or analytic algorithm by a human to explore alternative situations which 
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might occur. Rather than producing descriptive analyses of past performance, 
this form of output reflects a projection of what would occur in the future, 
based upon certain underlying relationships which govern behavior and inter- 
action of the system being examined and upon postulated assumptions regarding 
key characteristics of the future program environment. 

The performance of those transactions which comprise the day-to-day 
operational activities in the agency typically requires access to transaction 
documents, certain preplanned reports, and, from time to time, selected 
preplanned inquiries. These documents describe the current status of the 
system, salient characteristics which might influence decisions regarding 
the type of actions which need to be taken and alternative resources available 
which may be used in the transaction or event. 

To facilitate the management control process, information is required 
about planned performance, actual performance, and the variance from planned 
performance. This information is typically provided in preplanned reports 
and responses to preplanned inquiries and ad hoc reports and inquiries. The 
data base for management control consists of two major components: 1) descrip- 
tion of transactions and relevant system parameters, and 2) the performance 
criteria or standards to be achieved. 

Strategic planning entails the analysis of the likely impact upon the 
current program which changes in the current program environment will have, 
the potential impact of changes to the program if the environment remains 
the same, and combinations of both situations. To facilitate these analyses, 
information is required about the behavioral response patterns of the cur- 
rent program, the current program environment, and any changes to be con- 
sidered in the nature of the current program or the operating environment. 

B. AN APPROACH TO THE DESIGN OF A STATE VR AGENCY MIS 

The quality of information produced in the state VR agency and the 
extent, to which it is effectively used to support planning, management con- 
trol, and day-to-day events and transactions is strongly influenced by the 
MIS structure and operating procedures. When designing the MIS, each of 
the five system elements discussed in the previous section should be addressed. 
A successful system requires smooth operation of all five elements: if one 
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is inappropriately designed or functioning improperly, it is sufficient to 
jeopardize the effectiveness of the entire system. 

Frequently, the state agency MIS is the product of historical evolution 
and responses to agency procurement policies, staff capabilities, and other 
technical details, rather than a comprehensive systems design. The first 
step in the design or revision of a state VR agency MIS should be a thorough 
analysis of the agency's information needs; a framework for this analysis 
has been presented in Chapter II. Then, the individual components of the 
MIS should be designed to respond to these needs in the most effective and 
economical manner. 

A flow chart of the sequence of steps to be followed in the design of 
an MIS is presented in Figure 6. As illustrated, particular applications 
should be identified in terms of generic organizational activity and the 
organizational subsystem involved. Then, both the substantive content and 
descriptive characteristics, such as currency, accuracy, detail, and so forth, 
of the data needed to support these activities should be specified. Based 
upon these requirements, data presentation formats and report preparation 
time schedules should be established. Both the input data specifications 
and the required processing capabilities should be derived from the output 
specifications. Finally, a storage system adequate to keep the data required 
and to facilitate the necessary analyses should be selected. 

Now, many times the MIS design cannot, in fact, be carried out in the 
sequence illustrated in Figure 6. Data collection forms may have been de- 
signed for the agency years ago, a schedule of periodic reports may already 
exist, or the agency may be required to use an existing computer hardware 
system. However, the existence of constraints such as these should not 
cause all attempts at a logical approach to MIS design or revision to be 
abandoned. If the process diagrammed in Figure 6 is followed, it is fre- 
quently found that more can be done to design a responsive MIS, within the 
existing constraints, than was initially imagined. 
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Figure 6 

Flow Chart of Steps in the Design of a State VR Agency MIS 
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SOME OBSERVATIONS REGARDING STATE VR AGENCY 
MANAGEMENT INFORMATION SYSTEMS TODAY 

In response to RSA requirements and to the internal demands of program 
operations, all state VR agencies have developed systems for collecting and 
reporting selected program and fiscal data. While each system must collect 
a certain core group of data as specified by RSA, the data collection, stor- 
age, processing, and report generation activities have been developed in 
response to individual agency needs and capabilities. 

Review of a broad sample of the existing state VR agency information 
systems suggests that most agencies currently collect substantial amounts 
of program and fiscal data. However, it appears that the effectiveness 
with which this information is used to support program operations is varied. 
Following are observations on typical issues and concerns which characterize 
most state VR agency information systems: 

• Often, only a portion of data routinely collected is used 
in agency planning, management, or operations. 

• Field staff frequently question the validity of data upon 
which computerized reports are based. 

• Field staff often maintain manual files of client and fiscal 
data which they frequently use, despite the fact that the 
same data are recorded in a centralized data base and made 
available in computerized reports. 

• A majority of computerized reports are prepared to facilitate 
routine management control, a small number are designed for 
use in day-to-day operations, and a few, if any, are specific- 
ally designed to assist in planning activities. 

9 — Iir^eireYarrT^^rt - contervt - a^dTrormats are not aesigned~by 

identifying the particular need which they are to fulfill 
and consulting with the intended users to determine the infor- 
mation required; therefore 

— reports tend to be cluttered with a substantial amount 
of unnecessary information, thereby making it difficult 
to extract the data of importance; 
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— reports frequently are distributed to people for 
"information only 11 and are not really needed or used 
by these recipients in the performance of their jobs, 

• Most frequently, there is little or no attempt to highlight 
and interpret significant information presented in a report, 

o Relatively little capability exists to respond rapidly to 
particular information requests generated by agency managers 
and planners; as a result, little effort is made to use 
information interactively in the development of program 
plans and management approaches. 

© Selected efforts have been undertaken to develop creative 
insight into agency performance through the use of specially 
designed analytical schemes. One such attempt, the Caseload 
Analysis System, which develops assessments of key aspects of 
agency service delivery, is described in Attachments A and B 
to this report. 

• There exists in most state VR agencies the technological capa- 
bility to develop substantially improved systems for preparing 
information to facilitate effective planning, control, and 
operations at all levels. 

It is hoped that, through application of the concepts and approaches 
discussed in this paper, these and similar situations may be effectively 
resolved and state VR agency MIS's may be developed into streamlined tools 
to assist in the responsive provision of high-quality rehabilitation services. 
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ATTACHMENT A 

DESIGN AND IMPLEMENTATION OF A 
CASELOAD ANALYSIS SYSTEM FOR THE 
VIRGINIA DEPARTMENT OF REHABILITATIVE SERVICES 



Stanley E. Portny 
Stanley E. Portny and Associates, Inc. 
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DESIGN AND IMPLEMENTATION OF A CASELOAD ANALYSIS SYSTEM 
FOR THE VIRGINIA DEPARTMENT OF REHABILITATIVE SERVICES 

The Department of Rehabilitative Services has long been dedicated to 
providing rehabilitation services of the highest quality to eligible dis- 
abled people in Virginia. In its continuing effort to ensure the efficient 
and responsive delivery of services to clients, DRS has initiated a variety 
of activities aimed at increasing its ability to declare and achieve clear 
statements of program policy and uniform standards of service provision 
throughout the state. 

A key initiative in this effort is the development of a system for 
assessing case service provision. To accomplish this task, DRS awarded a 
contract to Stanley E. Portny and Associates; Inc, to design and assist in 
the implementation of a Caseload Analysis System (CAS). Different aspects 
of this activity included developing a methodology for assessing key aspects 
of service provision, specifying data required for completing these assess- 
ments, preparing specifications for required computer programming, and 
preparing a User's Manual and training materials. In addition, SEPA 
provided technical assistance during the introduction of thn CAS and with 
the evaluation of the reactions to its introduction. This rroject was 
initiated in 1980 and was completed by December 1981. 

PURPOSE AND STRUCTURE OF THE CAS 

The CAS is a structured approach for describing key aspects of the 
services provided to DkS* "clients. The purpose of the CAS is to help ensure 
that case services are provided in accordance with agency goals, objectives, 
and accepted agency standards of qiMlity. 

The CAS produces objective assessments of the extent to which casework 
is meeting established goals and criteria. This information is then fed 
back to service providers, to assist them in modifying their performance to 
conform with agency expectations; to agency managers, to help them to develop 
appropriate employee incentive systems and to assist them in the allocation 
of program resources; and to program policy makers. 
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The CAS entails three component operations. First, descriptive infor- 
mation regarding the case services provided during a particular time period 
is collected. Second, this information is assessed with respect to agency 
objectives to determine the relative adequacy of service provision activities. 
Third, the results of these analyses are condensed into succinct reports and 
distributed to agency administrators and service delivery personnel. 

Information generated by the CAS can facilitate improved case service 
delivery in several ways. First, it presents a timely statement to staff 
of the adequacy of their service provision activities with respect to the 
DRS mission, goals, and objectives, and, thereby, suggests to them particu- 
lar practices which they need to improve. Second, information from the CAS 
can be incorporated into staff appraisal and incentive systems, creating 
an added inducement for staff to perform in accordance with agency directives. 

Additionally, CAS analyses can help to identify areas where clarification 
and explanation of agency policies and procedures are required, as well as 
possible needs and topics for staff training. Information from the CAS can 
serve as one of the bases for allocating existing program resources and for 
predicting future resource needs. It can also be used to describe agency 
performance and to substantiate requests for funding and legislative actions 
at the state and federal levels. 

HIGHLIGHTS OF THE VIRGINIA CAS 

Following are highlights of the assessment scheme used in the Virginia 



CAS: 



• All cases closed in status 26 are examined. 

• No new data collection requirements are introduced. 

w" 4W^w*7«7HIWilb vjl wUJWAWttU 1IIUUU5V1IIWII t UXlU OOXVJL^G UCAJL V %ZJL J JL O 

based upon the priority category, cost of purchased services, 
time in status and outcome, including wages at closure and/or 
employment status of each case. 

• Performance is described with multiple measures, including a 
raw score and performance index for each individual factor 
and a composite index derived from a combination of the four 
individual indices . 
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• Performance assessments are determined for each case, and 
at the counselor, supervisor, regional, and state levels. 

• Reports are distributed quarterly to counselors, supervisors, 
regional and central office staff. 

KEYS FOR SUCCESS 

While the CAS inherently had the potential to significantly improve 
agency administration of the service delivery program, certain key factors 
substantially increased its chances for success: 

• The CAS was developed to respond to the interests and require- 
ments of the Client Services Division, and had the vigorous 
support of top DRS administrators. 

• The CAS incorporated reference performance criteria which 
related directly to the extent to which important agency ser- 
vice delivery objectives were met. 

• These criteria also reflected the relative priorities associated 
with the achievement of the agency's various goals and objec- 
tives. 

• Field staff were directly involved in the assessment of agency 
service delivery priorities. 

• The CAS was designed to make maximum use of program data 
already being collected and to place no additional reporting 
burdens on the field staff. 

• CAS reports were designed to present assessments in a clear 
and concise format. 

• Procedures were designed for using the results of CAS assess- 
menus inutner existing DRS management and evaluation systems 
and activities. In particular, the CAS was coordinated with 
policy planning and development, the DRS Management and Plan- 
ning System (MAPS), program evaluation, case reviews, and the 
DRS Management Information System. 
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ANCILLARY BENEFITS 



In addition to creating the capability for timely monitoring of vital 
aspects of service delivery activities, several additional benefits were 
realized as a result of the CAS development. First , attention was focused 
on specifying important agency goals and objectives in unambiguous and 
measurable terms. Though it was not claimed that the CAS criteria encom- 
passed all DRS goals, they did constitute clear statements of agency prior- 
ities in several key areas. Second, the surveying of DRS field staff during 
the CAS development offered the opportunity to ascertain field staff percep- 
tions of program and policy directives issued by the Central Office. This 
information provided valuable insight into the effectiveness of the current 
policy promulgation system. Third, field staff were made aware that program 
data which they had been collecting would, in fact, be used in a constructive 
way to assist them, in providing high quality services to clients. Conse- 
quently, there was an increased motivation for collecting this data and for 
insuring its accuracy and validity. 

Finally, it was anticipated that the CAS would serve as a vital link in 
the overall DRS communication system. For communication to be effective, 
the message must be stated in unambiguous terms and probes must be made to 
ensure that it had been received. The CAS would provide continuing inform- 
ation regarding the extent to which DRS policy statements had been received 
and acted upon and, thereby, inferences regarding the effectiveness of the 
communication process. Additionally, it offered field staff a vehicle for 
assessing the extent to which their efforts to respond to agency policy 
directives had been successful. 

FUTURE WORK 

A variety of areas have been explored for future expansion of the 
Virginia CAS. Among them are the following: 

• inclusion of cases closed in statuses 28, 30, and 08; 

• assessment of active cases; 

• inclusion of nonemployment outcome measures; 

• inclusion of intermediate outcomes; and 

• inclusion of new agency priorities as they arise. 
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ATTACHMENT B 

THE ILLINOIS DEPARTMENT OF REHABILITATION SERVICES 
CASELOAD ANALYSIS SYSTEM 
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THE ILLINOIS DEPARTMENT OF REHABILITATION SERVICES 
CASELOAD ANALYSIS SYSTEM 



A. WHAT IS THE CASELOAD ANALYSIS SYSTEM ? 

The caseload analysis system (CAS) is a set of procedures for describing 
the nature and quality of services provided to DORS clients. The purpose of 
the CAS is to help ensure that case services are provided in accordance with 
agency goals, objectives, and accepted agency standards of quality. 

B. HOW DOES THE CASELOAD ANALYSIS SYSTEM ACHIEVE ITS PURPOSE ? 

The CAS produces objective assessments of the extent to which casework 
is meeting established goals and criteria. This information can be used to 
identify performance which is particularly effective, as well as that which 
is problematic. Service delivery approaches contributing to the successful 
performance can be encouraged and shared. Practices which result in unaccep- 
table results can be studied and improved. 

Information generated by the CAS can be used to facilitate improved 
case service provision in several ways: 

• It can present a timely statement to staff of the adequacy of 
their service provision activities and, thereby, suggest to 
them particular practices which they should continue or areas 
in which they need to improve. 

o It can be incgrpgrat§d into Staff appraisal and incentive 
systems, thereby creating an added inducement for staff to 
perform in accordance with agency goals, objectives, and 
directives. 

t* It can help to identify areas where clarification and explan- 
ation of agency policies and procedures are required, as 
well as possible needs and topics for staff training. 

• It can serve as one of the bases for the allocation of existing 
program resources and for the prediction of future resource 
needs. 



© It can be used to describe agency performance and to substanti- 
ate requests for funding and legislative actions at the state 
and federal levels.' 

C. WHAT ACTIVITIES DOES THE CASELOAD ANALYSIS SYSTEM ENTAIL ? 

The CAS is comprised of three separate operations: 

• input , 

• processing, and 

9 OUtput. 

The input operation includes the gathering of all data required for the 
calculation of casework assessments. These data include statistics which 
describe each client served, the services provided, and the results of ser- 
vices, as well as performance levels against which casework will be assessed. 

Processing entails the calculations of performance assessments based 
upon the input data. These calculations are performed with the aid of the 
Central Office computer system. 

The output operation entails the preparation of reports detailing the 
results of the casework assessments and the dissemination of these reports 
to agency staff. A diagram illustrating the sequence of operations conducted 
in the CAS is presented below: 



Obtain caseload 
information and 
performance 
level targets 
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Analyze 

caseload 

activity 
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Input Process Output 



D. WHAT OUTPUTS ARE PRODUCED BY THE CASELOAD ANALYSIS SYSTEM ? 

Four reports will be produced by the CAS and distributed to DORS 
personnel: 

• district report, 

• area report, 
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• regional report, and 
0 state report. 

The district report presents CAS assessments for cases within the 
district, as well as average statistics for the district, area, region, and 
state. The area report presents a summary of the assessments for all dis- 
tricts within the area; the regional report summarizes CAS results for all 
areas within the region; and. the state report summarizes results for all 
regions within the state. 

E. WHAT INFORMATION IS REQUIRED FOR THE CASELOAD ANALYSIS SYSTEM ? 

All data required for CAS assessments are currently being collected by 
DORS. Consequently, the data will be obtained from existing agency files, 
and no additional data reporting requirements will be introduced . 

F. HOW IS CASEWORK ASSESSED IN THE CASELOAD ANALYSIS SYSTEM ? 

1. Which cases are assessed in the Caseload Analysis System? 

All cases closed in status 26 1 during the time period being considered 
will be included in the CAS assessment. This includes cases closed in 
status 26 and then reopened in status 32 during the same time period. 

2. Which characteristics of case service provision are considered in the 
Caseload Analysis System? 

Four factors are considered in the assessment of case service provision 

• priority — the relative emphasis placed on serving the target 
group in which the client is classified; 

• cost — the cost of purchased services in statuses 10-26; 

• time ~ the time which the client spends in statuses 10-26; and 

• outcome — the employment status of the client at closure. 



All status codes used in this manual are equivalent to those defined 
in the Illinois Department of Rehabilitation Services Client Services 
Manual, pp. 12-19. 
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3. How are the relationships between factor values and index scores 
determined? 

The index scores for priority and outcome which correspond to different 
priority groups and employment statuses, respectively, are determined annually 
based upon the opinions and judgments of DORS administrators. The relation- 
ships between index scores for cost and time and actual values of cost and 
time, respectively, are based upon agency program performance data for the 
last complete fiscal year. 

4. How are factor indices used to develop assessments of case service 
provision? 

Factor indices are interpreted in the following ways to yield a variety 
of measures of casework performance. For each case: 

• each factor index score is considered individually, and 

• a composite index is formulated from a weighted average of 
the individual factor scores. 

For groups of cases : 

e an average index score is calculated and considered for each 
factor, 

• an average composite index is calculated, 
0 a total composite index is calculated, and 

• an average total composite index is calculated. 

G. HOW DOES THE CAS RELATE TO OTHER DORS MANAGEMENT AND DATA SYSTEMS ? 

The following activities in support of agency management are currently 
performed in DORS: 

• quality assurance, 

• evaluation, and 

• policy planning and development. 

In addition, the STARS system serves as the primary mechanism for col- 
lecting and processing program data, while the Case Services Management 
System details selected approaches for using available program data. The 
CAS has been designed to complement, support, and build upon these other 
systems and processes, rather than to duplicate or replace them. 
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1. Quality Assurance 

Quality assurance activities entail the performance of detailed, 
periodic case reviews to assess compliance of casework with federal and 
state requirements, as well as the quality of services being provided. 
The CAS will generate information which can help to identify current or 
potential problems in service provision which can be examined more thoroughly 
through the conduct of appropriately designed case reviews. 

2. Evaluation 

Program evaluation entails the systematic observation of program activi- 
ties, the comparative assessment of program performance with respect to 
established goals, objectives, and other service criteria, and the formula- 
tion of recommendations for improving program performance, if required. 
Information produced by the CAS can serve as partial indicators of the effec- 
tiveness or efficiency of selected aspects of program operations to be con- 
sidered in the conduct of any program evaluation. Additionally, informatipn 
generated by the CAS can help to identify particular program components 
which should be examined in more detail through the conduct of a special 
evaluation study. 

3. Policy Planning and Development 

The development of responsive program policy requires the collection 
and consideration of data which accurately describe program elements of 
vital importance. Information produced by the CAS and subsequent attempts 
to determine reasons for demonstrated behavior and ways to improve problems 
discovered can serve to identify issues for which modification or develop- 
ment of DORS program policies should be considered. 

4. Statistical Tracking and Retrieval System 

The STARS is the main vehicle for collecting information from case 
records throughout the state and for processing and creating required sum- 
maries of this information. The majority of the reports currently produced 
by STARS are descriptive summaries of caseload movement , activity, and 
status at the time of report preparation. They are designed to provide 
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information to the staff to facilitate the making of appropriate and timely" 
decisions regarding service provision, as well as the appropriate monitoring 
of case progress. 

All data required for operation of the CAS are obtained from the STARS. 
Further, while reports generated by STARS are designed to describe current 
case service prpvision activity, CAS reports serve to assess and evaluate 
casework. 

5. Case Service Management System 

The CSMS is a set of procedures to be followed in the provision of case 
services. Specific guidance and directions are provided for all levels of 
DORS staff regarding particular activities for which they are responsible 
and ways in which these activities should be performed. All caseload data 
required to support the CSMS are obtained from the STARS. 

While 'the CSMS is a statement of the appropriate actions to follow, 
the CAS provides measures of the extent to which these actions have yielded 
the desired results. If problematic behavior is identified through the CAS, 
it is possible that the CSMS may detail procedures which might help to im- 
prove the situation. 

H. WHAT CONTINUING ACTIVITIES RELATED TO THE CASELOAD ANALYSIS SYSTEM 
WILL BE CONDUCTED ? 

Two types of activities will be conducted on a continuing basis: 

• updating of parameter values used to determine the factor 
index scores, and 

• evaluation and improvement of the CAS assessment methodology. 

The parameters used to define the relationships between values of 
priority, cost, time, and outcome and their respective index scores are 
determined from agency performance data and the judgments . and opinions of 
DORS administrators. At the start of each fiscal year, performance statis- 
tics from the recently completed fiscal year are used to update the schemes 

for determining the cost and time factors. At the same time, a structured 

o 

questionnaire will be used to solicit input from DORS administrators to up- 
date the index scores assigned for particular priority groups and employment 



statuses. The results of these activities will be presented in a memorandum , 
together with required modifications to this User's Manual, which will be 
distributed to agency staff. 

Each year, a review will be made of the extent to and manner in which 
CAS information was used, elements which might be added or deleted from the 
case service assessment scheme and any other benefits or problems which 
resulted from the operation of the CAS, Plans for modifying the CAS based 
upon the results of this review will be formulated, and required changes in 
CAS structure and/or operating procedures will be introduced at the start 
of the new fiscal year. 
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